Anytime Guarantees for Reachability in Uncountable Markov Decision Processes

We consider the problem of approximating the reachability probabilities in Markov decision processes (MDP) with uncountable (continuous) state and action spaces. While there are algorithms that, for special classes of such MDP, provide a sequence of approximations converging to the true value in the limit, our aim is to obtain an algorithm with guarantees on the precision of the approximation. As this problem is undecidable in general, assumptions on the MDP are necessary. Our main contribution is to identify sufficient assumptions that are as weak as possible, thus approaching the "boundary" of which systems can be correctly and reliably analyzed. To this end, we also argue why each of our assumptions is necessary for algorithms based on processing finitely many observations. We present two solution variants. The first one provides converging lower bounds under weaker assumptions than typical ones from previous works concerned with guarantees. The second one then utilizes stronger assumptions to additionally provide converging upper bounds. Altogether, we obtain an anytime algorithm, i.e. yielding a sequence of approximants with known and iteratively improving precision, converging to the true value in the limit. Besides, due to the generality of our assumptions, our algorithms are very general templates, readily allowing for various heuristics from literature in contrast to, e.g., a specific discretization algorithm. Our theoretical contribution thus paves the way for future practical improvements without sacrificing correctness guarantees

An Anytime Algorithm for Reachability on Uncountable MDP

We provide an algorithm for reachability on Markov decision processes with uncountable state and action spaces, which, under mild assumptions, approximates the optimal value to any desired precision. It is the first such anytime algorithm, meaning that at any point in time it can return the current approximation with its precision. Moreover, it simultaneously is the first algorithm able to utilize \emph{learning} approaches without sacrificing guarantees and it further allows for combination with existing heuristics

Distribution patterns of 104 kDa stress-associated protein in rice

10.1023/A:1006099715375Plant Molecular Biology376911-919PMBI

Semantic Abstraction-Guided Motion Planningfor scLTL Missions in Unknown Environments

Complex mission specifications can be often specifiedthrough temporal logics, such as Linear Temporal Logic and itssyntactically co-safe fragment, scLTL. Finding trajectories thatsatisfy such specifications becomes hard if the robot is to fulfilthe mission in an initially unknown environment, where neitherlocations of regions or objects of interest in the environmentnor the obstacle space are known a priori. We propose an algorithmthat, while exploring the environment, learns importantsemantic dependencies in the form of a semantic abstraction,and uses it to bias the growth of an Rapidly-exploring randomgraph towards faster mission completion. Our approach leadsto finding trajectories that are much shorter than those foundby the sequential approach, which first explores and then plans.Simulations comparing our solution to the sequential approach,carried out in 100 randomized office-like environments, showmore than 50% reduction in the trajectory length.QC 20210803</p